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A Human-In-The-Loop experiment was conducted at the National Aeronautics and Space Administration (NASA) 
Langley Research Center (LaRC) to investigate the viability of the In-Trail Procedure (ITP) concept from a flight 
crew perspective, by placing participating airline pilots in a simulated oceanic flight environment. The test subject 
pilots used new onboard avionics equipment that provided improved information about nearby traffic and enabled 
them, when specific criteria were met, to request an ITP flight level change referencing one or two nearby aircraft 
that might otherwise block the flight level change. The subject pilots’ subjective assessments of ITP validity and 
acceptability were measured via questionnaires and discussions, and their objective performance in appropriately 
selecting, requesting, and performing ITP flight level changes was evaluated for each simulated flight scenario. 
Objective performance and subjective workload assessment data from the experiment’s test conditions were 
analyzed for statistical and operational significance and are reported in the paper. Based on these results, suggestions 
are made to further improve the ITP. 

Introduction 

Aircraft in oceanic and remote non-radar airspace 
frequently fly for extended periods of time in the 
same direction and along a limited number of pre- 
defined flight paths. Since there is no radar 
surveillance and only limited direct communications 
available, controllers use procedural separation rules 
to ensure that aircraft remain separated. Procedural 
separation typically requires much greater time or 
distance between aircraft than when radar 
surveillance and direct communications are available. 

This limits the number of aircraft on a given track at 
a given altitude or flight level (FL). 

Furthermore, an aircraft’s most fuel-efficient altitude 
is lower during the early segments of a flight, when 
the aircraft has a heavy fuel load, and higher during 
the later segments when much of the fuel has been 
consumed; so operational efficiency is generally 
enhanced by climbing to a higher flight level one or 
more times during a long flight segment. However, in 
many situations the standard longitudinal separation 
interval does not exist at neighboring, or intervening, 
flight levels but may exist at a succeeding flight 
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level. An aircraft desiring a flight level change to 
such a succeeding flight level would therefore be 
prevented or “blocked” from making the climb or 
descent through the intervening flight level(s) (Figure 
1 ). 



Figure 1. Aircraft at Flight Level 340 (FL340) desires 
a climb to FL360 

A new In-Trail Procedure (ITP) was developed to 
enable climbs and descents through normally blocked 
flight levels, providing a safe and practical method 
for Air Traffic Service Providers (ATSP) to approve, 
and flight crews to conduct, such operations. This 
procedure was developed under the Enhanced 
Oceanic Operations (EOO) research element as part 
of the NASA Next Generation Air Transportation 


System (NextGen) Air Traffic Management (ATM) 
Airspace Project. EOO’s objective was to develop 
methodologies, concepts, and procedures to reduce 
separation requirements for future air transportation 
systems using airborne Automatic Dependent 
Surveillance-Broadcast (ADS-B) and Airborne 
Separation Assistance Systems (ASAS). EOO 
research was specifically focused on developing a 
globally accepted, early application of airborne ADS- 
B that results in more efficient oceanic and remote 
non-radar operations while providing opportunities 
for research of ASAS. This early application should 
provide operational expertise with ASAS and be an 
incentive for operators to voluntarily upgrade fleet 
equipment enabling system-wide transformation. 

In-Trail Procedure 

The ITP is described in the Operational Services and 
Environment Description (OSED) document [1] from 
the RTCA/EUROCAE Requirements Focus Group 
(RFG). The procedure requires the crew to use ADS- 
B information derived onboard the aircraft to 
determine if criteria required for an ITP are met. The 
aircraft-derived information includes flight identifier 
(ID), flight level, same direction of travel, ITP 
distance, and ground speed differential (all relative to 
potentially blocking aircraft). The equipment onboard 
the ITP enabled aircraft receives the ADS-B data and, 
along with onboard navigation data, calculates 
appropriate separation information for these aircraft 
and portrays this information to the crew. Using this 
information, the crew determines whether to request 
a standard or ITP flight level change. If the desired 
flight level appears available but potentially blocking 
aircraft are observed at the intervening flight levels 
and the requested flight level is no more than 4,000 
feet (ft.) from the initial flight level, then the crew 
would evaluate the available information for these 
potentially blocking aircraft to determine if they can 
be used as reference aircraft in an ITP flight level 
change request. An aircraft at an intervening flight 
level can be used as a reference aircraft if it meets the 
following criteria: 

Same direction of flight as the ITP Aircraft, 

+/- 45 degrees, 

Qualified ADS-B data are being received 

from the aircraft, and 

ITP distance/speed criteria are met. 

The equipment onboard the ITP enabled aircraft 
calculates for each aircraft the ITP distance, which is 
the difference in distance to a common point as 
shown in Figure 2 (for identical ground tracks, ITP 
distance is simply the distance between the two 


aircraft). The ITP speed criterion is a ground speed 
differential, also calculated by the equipment onboard 
the ITP enabled aircraft and is simply the difference 
in ground speed between the two aircraft. A positive 
ground speed differential is one which results in a 
decreasing ITP distance between the respective 
aircraft. 



Figure 2. ITP distance 

A reference aircraft must meet the following ITP 
distance/speed criteria, which were established 
during safety analyses of the procedure: 

ITP Distance at least 15 nautical miles 
(nmi), and positive ground speed differential 
of 20 knots (kts) or less, or 
ITP Distance at least 20 nmi, and positive 
ground speed differential of 30 kts or less. 

Up to two reference aircraft can be included in an 
ITP flight level change request to ATSP. All possible 
ITP geometries are permissible as long as all aircraft 
meet the criteria for reference aircraft. 

In the simulation study an ITP flight level change 
was requested through a simulated data-link 
interface, similar to that required to make a standard 
flight level change request, with additional ITP- 
specific information entered on the available free text 
lines. The keyword “ITP” had to start the first (and 
only the first) free text line followed by the 
information of each of the one or two reference 
aircraft, according to the following format: 

F/<reference aircraft flight id>/nn or 
L/<reference aircraft flight id>/nn 

Where: 

F/ means following this reference aircraft 
L / means leading this reference aircraft 
/nn is the ITP Distance, in nmi 


Given that the ITP is an airborne traffic situation 
awareness application there is no change in the 
responsibilities of either pilots or controllers. The 
flight crew continues to be responsible for the 
operation of the aircraft and conformance to 
clearances, and the controller continues to be 
responsible for separation and the issuance of 
clearances. However, the controller would use 
additional information transmitted in the ITP 
aircraft’s request to determine if separation can be 
assured and a clearance can be issued. 

Upon receipt of an ITP request, ATC would 
determine if a standard (non-ITP) flight level change 
clearance could be issued. If a non-ITP clearance is 
not possible, the controller would verify that the 
reference aircraft have not been cleared to change 
speed or flight level, are not about to reach a point at 
which a significant change of track will occur, and 
have a positive Mach difference of no greater than 
0.04 relative to the requesting aircraft 6 , If these ITP 
ATC criteria are met and separation exists at the 
requested flight level with other aircraft, ATC may 
issue the ITP flight level change clearance. 

If an ITP clearance is received, then the crew must 
reassess the reference aircraft identified in the 
clearance to assure that the ITP distance/speed 
criteria are still met before accepting the clearance. If 
the criteria are no longer met, then the clearance must 
be declined. 

Once an ITP clearance has been accepted, the crew 
should commence the flight level change without 
delay and maintain cruise Mach number and at least 
300 feet per minute (FPM) vertical speed throughout 
the flight level change. If this minimum performance 
cannot be maintained, then the applicable ICAO 
defined regional contingency procedures for inability 
to conform to an ATC clearance must be followed. 

The ITP aircraft crew is not required to monitor the 
ITP distance to the reference aircraft during the climb 
or descent. The separation assurance of the ITP is 
ensured by the proper initial conditions and by 
maintenance of the minimum required vertical speed 
during the maneuver. The ITP is completed when the 
ITP flight crew reports being established at the new 
flight level. 


6 ATC checks the difference in (airspeed-based) 
assigned Mach numbers rather than groundspeed 
differential in knots as checked by the flight crew. 
The Mach check is required by the safety case to 
account for non-uniform winds at different flight 
levels. 


Study Objectives and Hypotheses 

The first objective of the EOO Human-In-The-Loop 
ITP Validation Simulation Study was to assess the 
validity of the ITP by determining if subject pilots: a) 
were able to perform ITP maneuvers during 
simulated oceanic flights, and b) found the 
procedural steps that they were instructed to use 
while executing ITP maneuvers to be correct, 
complete, and appropriately specified. 

The second objective was to assess pilot acceptability 
of the ITP. In addition to collecting subject pilots’ 
impressions of the overall acceptability of the ITP, 
the study attempted to determine if subject pilots 
found that: a) the level of workload that they 
experienced while performing ITP maneuvers was 
acceptable, and b) the ITP will likely provide 
perceived benefits possibly in the form of improved 
traffic situation awareness, smoother rides and/or fuel 
savings. 

It was hypothesized that subject pilots would be able 
to perform ITP maneuvers (i.e., that the instructed 
procedural steps would be performed correctly and in 
the appropriate order) and that subject pilots would 
not find any missing, incomplete, or extraneous 
procedural steps associated with the ITP. 
Additionally, it was hypothesized that subject pilots 
would find the workload level associated with 
performing ITP flight level changes to be acceptable; 
that is, subject pilots were expected to provide 
workload ratings of “3” or less using the Modified 
Cooper-Harper (MCH) Rating Scale which ranges 
from 1 to 10 (1 = Minimal Operator Effort, 3 = 
Medium Operator Effort, 7 = Maximum Operator 
Effort, and 10 = Task can not be accomplished) [2], 

Note that this research endeavor only studied ITP 
validity and acceptability to pilots. Validity and 
acceptability to controllers were not considered in 
this experiment. 

Participants 

Participants consisted of 12 commercial airline pilots 
with current oceanic experience. All of the subject 
pilots were male and ranged in age between 42 - 59 
years [Mean (M) = 49, Standard Deviation ( SD ) = 6]. 
Five of the participants were captains, and the other 
seven were first officers. On average, the participants 
had 18 years of airline experience ( M = 18.6, SD = 
7.2) and over 9,000 hours of airline flying experience 
(M = 9,892, SD = 5,829). All of the participants had 
experience with data-link communications and 
Automatic Dependent Surveillance-Contract (ADS- 



C). Two of the participants had previous experience 
using Electronic Flight Bag (EFB) devices, and one 
participant had previous experience using a Vertical 
Situation Display (VSD). 

Test Facilities and Apparatus 

The aircraft simulation used for this experiment, 
known as the NASA LaRC Aircraft Simulation for 
Traffic Operations Research (ASTOR) [3], is a 
medium-fidelity computer workstation-based desktop 
flight simulator whose displays and control panels are 
representative of a current-generation generic 
transport aircraft flight deck (e.g., a B-777). 

In addition to the main ASTOR displays and control 
panels, the pilot interface for the experiment also 
included an EFB simulation. This EFB simulation 
hosted the ITP application and was run on a separate 
tablet personal computer (PC) to the left of the two 
ASTOR screens (see Figure 3). Interaction with the 
ITP application was accomplished by the subject 
pilots using a separate computer mouse from the one 
used to interact with the main ASTOR displays and 
control panels. 



Figure 3. ASTOR layout 


The ITP application consisted of a newly designed 
interface and a prototype implementation of the logic 
contained in the OSED document describing the ITP 
maneuver constraints. This ITP algorithm logic, 
described in a formal specification language [4], was 
used to determine the status of many of the elements 
on the ITP interface display (figure 4 Figure 4 ). 
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Figure 4. ITP application symbology 


A desktop flight simulation program called the 
Traffic Manager (TMX) performed the function of 
background traffic generator and ATC simulation. 
TMX was originally developed by the National 
Aerospace Laboratory (NLR) in The Netherlands, 
and it can also serve as a stand-alone traffic 
simulator, flight simulation scenario generator and 
editor, experiment control station, data-recording 
tool, and rapid prototyping environment [5]. 


Experiment Design 


The experiment design used for data collection was 
an 8 (Type of maneuver) x 2 (Ability to complete 
maneuver) full-factorial, within-subject design (Table 
1). The twelve subject pilots (Si_i 2 ) performed all 16 
test conditions in random order. 


Table 1. Experiment Design Matrix 


Type of maneuver 

Able to 
complete 

Not able to 
complete 

Standard Climb 

Sl-12 

Sl-12 

Standard Descent 

Sl-12 

Sl-12 

ITP Following Climb 

Sl-12 

Sl-12 

ITP Following Descent 

Sl-12 

Sl-12 

ITP Leading Climb 

Sl-12 

Sl-12 

ITP Leading Descent 

Sl-12 

Sl-12 

ITP Combined Climb 

Sl-12 

Sl-12 

ITP Combined Descent 

Sl-12 

Sl-12 


Note: The reference aircraft is located behind the ITP 
aircraft during a leading maneuver, and ahead of the 
ITP aircraft during a following maneuver. 


A flight level change was considered “Not able to 
complete” if either the target and/or intermediate 
flight level was blocked by traffic aircraft of which 
the subject pilot was unaware (i.e., controller rejects 
request) or blocked by traffic aircraft of which the 
subject pilot was aware (i.e., pilot should not request 
flight level change). 


The dependent measures for the experiment included 
selection errors, execution errors and subjective 
assessments of the ITP’s validity, acceptability and 




workload. Selection errors occurred when subject 
pilots made a flight level change request that was 
different from the expected request for a given 
scenario. Execution errors occurred if subject pilots: 
a) failed to correctly communicate required 
information to ATC while making either a standard 
or an ITP flight level change request, or b) failed to 
adhere to the aircraft performance criteria required 
during ITP flight level change maneuvers. Depending 
on the potential consequences associated with a given 
execution error, it may have been identified as a 
“safety-related execution error.” 

The subject pilots’ perceptions regarding the validity, 
acceptability and workload of the ITP were collected 
using: 1) a post-scenario questionnaire that was 
administered after the completion of each test 
condition, 2) a post-experiment questionnaire that 
was administered after the completion of the final test 
condition, and 3) a post-experiment group debrief 
session. 


an ITP flight level change when they were expected 
to have realized that an ITP maneuver was not 
possible because an observable traffic aircraft was 
blocking the ITP maneuver. Seven ITP selection 
errors occurred when subject pilots included an 
additional reference aircraft (located at the desired 
altitude) in their request for an expected leading or 
following ITP scenario. One ITP selection error 
involved a subject pilot failing to reference both the 
leading and following aircraft in what was expected 
to be a combined ITP flight level change. The 
remaining ITP selection error involved a subject pilot 
requesting a standard flight level change when he 
was expected to determine that a standard flight level 
change was not possible due to blocking aircraft, nor 
was an ITP maneuver possible because the ground 
speed differential of the blocking aircraft did not 
meet the required ITP speed criteria for a reference 
aircraft. 

Execution Errors 


Results 

Subject pilots participated in the study in groups of 
three to six, with each group participating in the 
study over the course of two consecutive days. 
Approximately the first half day consisted of a 
combination of “classroom” and hands-on ITP and 
experiment procedures training. Collectively, the 
subject pilots performed 192 simulated flight 
scenarios while completing the experiment’s 16 test 
conditions. The 192 simulated flight scenarios 
consisted of 48 (4 standard scenarios x 12 subject 
pilots) standard flight level changes and 144 (12 ITP 
scenarios x 12 subject pilots) ITP flight level 
changes. 

Selection Errors 


Execution errors were evaluated after subject pilots 
chose to request a given type of flight level change 
maneuver. As a result of the subject pilots’ selection 
errors, there was a change in the total number of 
standard scenarios and ITP scenarios during which 
execution errors could occur. Of the original 48 
expected standard scenarios, six were “converted 
into” ITP scenarios, and one ITP selection error 
resulted in the conversion of an expected ITP 
scenario into a standard scenario. Therefore, subject 
pilots requested a total of 43 standard flight level 
changes and 149 ITP flight level changes (Table 2). 


Table 2. Expected versus Requested Scenarios 



Standard 

ITP ! 

Expected 

Requested 

Expected 

Requested 

Total 

48 

43 

144 

149 


The subject pilots made 19 selection errors during the 
192 scenarios, yielding an overall selection error rate 
of 9.9%. During the 48 expected standard scenarios, 
subject pilots made a total of six ITP flight level 
change requests. That is, 12.5% of the time subject 
pilots requested an ITP flight level change when they 
were expected to request a standard flight level 
change. However, the subject pilots may have been 
biased towards making ITP requests since they were 
instructed during their training: “when in doubt 
[about aircraft spacing], request an ITP.” 

Thirteen selection errors occurred during the 144 
expected ITP scenarios resulting in an ITP selection 
error rate of 9.0%. Four of the 13 ITP selection errors 
involved instances in which subject pilots requested 


Subject pilots did not commit any execution errors 
during the 43 requested standard scenarios. 

During 29 of the 149 (19.5%) requested ITP 
scenarios, subject pilots committed at least one 
execution error. Since it was possible for a subject 
pilot to make multiple errors during a single scenario, 
a total execution error count of 33 occurred. Of the 
total execution error count, nine were identified as 
involving safety-related execution errors. In general, 
two types of errors were classified as being safety- 
related: 1) failure to adhere to performance 

requirements, and 2) the inclusion of inaccurate 
information in a flight level change request that could 
either not be verified by ATC or not be evaluated by 
ATC as containing inaccuracies. 




Two of the safety-related ITP execution errors 
occurred when subject pilots identified a leading 
aircraft as a following aircraft. Two safety-related 
ITP execution errors occurred when subject pilots 
failed to include the third number of a reference 
aircraft’s call sign, and three safety-related ITP 
execution errors occurred when subject pilots 
referenced an incorrect aircraft. The remaining two 
safety-related ITP execution errors occurred when 
one subject pilot failed, in two separate scenarios, to 
adhere to the aircraft performance criteria required 
during a requested ITP scenario. In one instance, the 
subject pilot made a deliberate speed change of 0.01 
Mach that allowed him to perform a maneuver that 
otherwise would not have been allowed; in the other 
instance, the same subject pilot failed to maintain 
required vertical speed. 

A McNemar Test (i.e., a nonparametric within- 
subject test appropriate for analyzing two related 
samples of nominal data) [6] showed that 
significantly fewer safety-related execution errors 
occurred during the requested standard scenarios (i.e., 
no errors) than during the requested ITP scenarios (p 
<0.01). 

Twenty-four “non-safety-related execution errors” 
occurred during the scenarios involving ITP flight 
level changes. Sixteen ITP execution errors involved 
instances in which subject pilots referenced an 
aircraft at the desired altitude during a flight level 
change request; three ITP execution errors involved 
subject pilots requesting a climb or descent to an 
incorrect altitude; three ITP execution errors were 
related to syntax errors in the ITP phraseology; one 
ITP execution error occurred when a subject pilot 
failed to comply with the ATC instruction to “report 
reaching” when level at the completion of the 
approved maneuver (per ICAO procedures for 
oceanic/remote flight operations); and one ITP 
execution error involved a subject pilot failing to 
accept ATC’s denial of an ITP flight level change 
request (per the briefed standard procedures for data- 
linked ATC communications) before making a 
second ITP flight level change request. 

Subjective Assessments of the ITP’s Validity 

The subject pilots indicated in 92.7% of their post- 
scenario questionnaire responses that the procedural 
steps used to execute ITP maneuvers were correct, 
complete, and appropriately specified. Suggestions 
were made to improve the checklist by including 
additional steps (i.e. confirmation steps between Pilot 
Flying and Pilot Not Flying) to prevent oversight and 
to avoid syntax errors. Two of the subject pilots 


indicated they used the ITP checklist during every 
ITP maneuver while, on average, the other 10 subject 
pilots reported using the checklist during 37% ( SD = 
17.4%) of the ITP maneuvers. When asked if they 
reassessed the ITP criteria every time they received 
an ITP clearance, as required by the checklist, eight 
of the subject pilots stated that they always 
performed the required reassessment while four of 
the subject pilots stated that they did not adhere to 
this procedural step. When asked to describe their 
ITP performance concerns, the subject pilots 
mentioned the existence of wake vortices during the 
execution of an ITP maneuver; however, it was 
acknowledged that the impact of wake vortices must 
also be considered during current day operations. 
Second, the potential consequences associated with 
failures of aircraft to comply with ITP performance 
requirements were identified as a cause for concern. 

Subjective Assessments of the ITP’s Acceptability 

All 12 subject pilots found the ITP to be an 
acceptable procedure, but two subject pilots 
mentioned concerns regarding the procedure’s 
reliance on an “honor system” in which the distances 
among aircraft must be accurately reported to ATC. 
Two subject pilots indicated that the ITP’s 
acceptability would increase if pilots were afforded 
the opportunity to communicate alternate altitude 
requests to ATC in hopes of securing approval for a 
flight level change should their initial requests be 
denied. With respect to their impressions regarding 
the level of safety associated with performing the ITP 
as compared with current day procedures, one subject 
pilot characterized the ITP as being “less safe than” 
current day procedures but stated that the ITP is still 
“procedurally sound”; seven subject pilots 
characterized the ITP as being “equally as safe as” 
current day procedures; and four subject pilot 
characterized the ITP as being “safer than” current 
day procedures. Subject pilots indicated that the 
improved safety was a direct result of the enhanced 
situation awareness (SA) provided by the ITP 
application’s display. When asked to comment on 
their use of free text phraseology to communicate 
with “ATC,” subject pilots indicated that, in general, 
they found the terminology to be straightforward and 
easy to learn and felt that “Leading” and “Following” 
were good terms to use since it is logical for pilots to 
reference their ITP Aircraft. It was pointed out, 
however, that airlines often discourage the use of free 
text due to language barriers and the possibility of 
misinterpretation. Therefore, one subject pilot 
suggested that the ITP display provide “clickable” 
information that could be used to populate free text 
fields to reduce typographical errors and/or language 



differences (e.g., ITP climb/descend could be 
selectable). When asked about potential benefits that 
the ITP might have, the subject pilots identified: 
improved traffic SA; more efficient aircraft operation 
(resulting from the ability to manage climbs at 
efficient times); enhanced job satisfaction associated 
with the opportunity to achieve optimum 
performance; and improvements in safety due to the 
avoidance of weather and turbulence. 

In addition to providing feedback regarding the ITP’s 
overall acceptability, safety, and phraseology, the 
subject pilots also shared enlightening viewpoints 
regarding ATC’s role in separation assurance. 
Several of the subject pilots indicated that if the 
appropriate information is made available to flight 
crews, then the level of responsibility for separation 
assurance that is given to pilots during ITP 
maneuvers is acceptable. However, when the 
potential for increasing pilot responsibility for 
separation assurance was considered, one commonly 
expressed opinion was that pilots do not want more 
responsibility for separation assurance, particularly 
because of “violation potential,” and that pilots have 
“no interest in being an air traffic controller.” 

Subjective Assessments of the ITP’s Workload 

After completing each of the experiment’s 16 test 
conditions, subject pilots used the MCH Rating Scale 
to report the level of workload that they had just 
experienced. 
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Figure 5. Mean MCH workload ratings 

The MCH workload ratings associated with the 
requested standard scenarios revealed a mean of 1.20 
[SD = 0.40, Sample Size N = 43], and the MCH 
workload ratings associated with the requested ITP 
scenarios revealed a mean of 1.58 (SD = 0.75, N = 
149) (figure 5 Figure 5 ). A Wilcoxon Test (i.e., a 
nonparametric within-subject test appropriate for 
analyzing two related samples of ordinal data) [7] 
revealed that a significantly higher workload rating 


was associated with the requested ITP flight level 
change maneuvers compared to requested standard 
flight level change maneuvers (p < 0.01). It is 
asserted, however, that the difference is not 
operationally significant. Subject pilots indicated that 
other phases of flight (i.e. descent and approach) 
have much higher workload ratings than ITP flight 
level change maneuvers. 

Discussion 

When the overall results are examined, it is apparent 
that in the majority of scenarios, subject pilots were 
able to correctly assess the traffic situation, select an 
appropriate response, and execute their selected flight 
level change procedure, if any, without error. 
However, it could also be asserted that the error rates 
for ITP maneuvers were higher than for standard 
flight level changes, and that these errors have design 
implications for both the ITP and the prototype traffic 
display. Nine of the 33 execution errors were 
specified as “safety-related” (errors that potentially 
cannot be detected by ATC). These errors, however, 
might be mitigated by: 1) improving the prototype 
ITP operation’s display interface and symbology; 2) 
limiting the applicability of the ITP to well-defined 
simple geometries; 3) defining specific rules-of- 
thumb (versus the current judgment-based guidance) 
as to when pilots should request an ITP; and/or 4) 
increasing the level of automation/decision support 
provided by the ITP operations application. 

There was also an instance where the subject pilot 
“gamed” the scenario by intentionally changing his 
speed by 0.01 Mach to allow an ITP that would not 
otherwise have been possible. The possibility of 
pilots “gaming” such situations was widely discussed 
during the debrief sessions with the subject pilots, 
and the pilots’ consensus was that such behavior 
would inevitably occur occasionally, at some 
unknown frequency, but would not be considered 
hazardous to safety. Note, however, that oceanic 
clearances typically require that speed be maintained 
to within 0.01 Mach to maintain longitudinal 
separation in non-radar environments. 

Sixteen of the 24 non-safety-related execution errors 
(i.e., two-thirds of these errors) involved subject 
pilots incorrectly referencing aircraft at the desired 
final, rather than the intervening, flight level. Why 
the above -described errors occurred, and especially 
why they occurred in such large numbers, is a source 
of significant puzzlement and speculation among the 
authors. During the debrief sessions subject pilots 
were asked for possible explanations. The answers 
varied from: either the subject pilots didn’t realize 


they had made the error and couldn’t say why they 
had done so; they did it by mistake and recognized 
afterward that they had done so; or they intentionally 
did it to let ATC know that, for example, they “knew 
about the presence of that aircraft.” 

Conclusions 

Analysis of both the qualitative and quantitative data 
revealed that the ITP, from a pilot’s perspective, is 
generally both valid and acceptable within the 
conditions of the experiment. Although the workload 
level reported for the ITP flight level change requests 
was higher than the workload level reported for the 
standard flight level change requests, subjective 
workload assessments were well within acceptable 
limits. In addition, the subject pilots’ subjective 
acceptability ratings and comments regarding the ITP 
were generally high and positive, respectively. While 
the ITP was generally regarded to be valid and 
acceptable, the error rates for ITP maneuvers were 
higher than for standard flight level changes, but 
mitigating strategies were suggested. 

In the end, the subject pilots found the procedure 
valid, acceptable, “procedurally sound,” and 
“intuitively obvious” and were very enthusiastic 
about the enhanced SA provided by the ITP 
operations application display. The subject pilots 
were somewhat unenthusiastic regarding the notion 
of more pilot responsibility for separation assurance 
in future applications, but would like to see the 
presented ITP implemented during current day 
oceanic flight operations. 
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